Skip to content

fix(assertions): render collection contents in IsEqualTo failure messages (#5613 B)#5619

Merged
thomhurst merged 3 commits intomainfrom
fix/5613-array-equals-format-fallback
Apr 18, 2026
Merged

fix(assertions): render collection contents in IsEqualTo failure messages (#5613 B)#5619
thomhurst merged 3 commits intomainfrom
fix/5613-array-equals-format-fallback

Conversation

@thomhurst
Copy link
Copy Markdown
Owner

Summary

Test plan

  • Issue5613ArrayFormatTests: 5 regression tests (array contents differ, array contents match + reference differ → hints IsEquivalentTo, int array, List<int>, large array truncation)
  • Full TUnit.Assertions.Tests suite passes across net472, net8.0, net9.0, net10.0
  • TUnit.PublicAPI snapshot tests pass (no public API changes)

…ages (#5613 B)

Arrays and List<T> fall back to reference equality in EqualityComparer<T>.Default
and don't override ToString, so failures surfaced as "System.Int32[]" vs
"System.Int32[]". Failures now show element contents (capped at 10 items) and,
when contents sequence-equal but references differ, point users at
IsEquivalentTo to clarify why a visually-equal assertion failed.
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: fix(assertions): render collection contents in IsEqualTo failure messages

The fix is directionally correct and addresses a genuine UX pain point — the before/after improvement in failure message clarity is significant. Tests are well-structured and cover the important edge cases. One critical bug and one derived issue need attention before merge.


Critical: Double Enumeration in CheckAsync

File: TUnit.Assertions/Conditions/EqualsAssertion.cs

var actualPreview = FormatValue(value);          // first enumeration
if (SequenceEqualsNonGeneric(value!, _expected!)) // second enumeration of value + expected

FormatValue exhausts the enumerator for value, then SequenceEqualsNonGeneric tries to enumerate value again. For non-replayable sequences — LINQ queries, yield return generators, File.ReadLines(), IAsyncEnumerable-wrapped sources — the second pass sees an empty sequence. This causes incorrect "same contents, different reference" diagnostics even when the contents are completely different.

The same issue applies to _expected: FormatValue(_expected) in GetExpectation() runs in a separate call, potentially after CheckAsync has already consumed _expected via SequenceEqualsNonGeneric.

Better approach — buffer once, reuse:

if (_comparer is null && IsFormattableCollection(value) && IsFormattableCollection(_expected))
{
    var actualItems  = Materialize(value!);
    var expectedItems = Materialize(_expected!);
    var actualPreview = FormatItems(actualItems);

    if (SequenceEqualsItems(actualItems, expectedItems))
        return Task.FromResult(AssertionResult.Failed(
            $"received {actualPreview} (same contents, different reference — use IsEquivalentTo to compare by contents)"));

    return Task.FromResult(AssertionResult.Failed($"received {actualPreview}"));
}

// ---
private static List<object?> Materialize(object value)
{
    var list = new List<object?>();
    foreach (var item in (IEnumerable)value) list.Add(item);
    return list;
}

private static bool SequenceEqualsItems(List<object?> a, List<object?> b)
{
    if (a.Count != b.Count) return false;
    for (int i = 0; i < a.Count; i++)
        if (!Equals(a[i], b[i])) return false;
    return true;
}

private static string FormatItems(List<object?> items)
{
    var preview = string.Join(", ", items.Take(CollectionPreviewMax).Select(x => x?.ToString() ?? "null"));
    if (items.Count > CollectionPreviewMax)
        preview += $", and {items.Count - CollectionPreviewMax} more...";
    return $"[{preview}]";
}

Why is buffering the right call? It eliminates the double-enumeration hazard at the cost of a single extra allocation in the failure path — which is already allocating a string, so the relative cost is negligible. Arrays and List<T> (the common cases) aren't affected by the current bug, but the correct solution is still to enumerate once.


Minor: FormatValue iterates the full collection for the truncation count

foreach (var item in enumerable)
{
    if (total < CollectionPreviewMax) items.Add(...);
    total++;   // continues counting after cap
}

A 10,000-element array will iterate all 10,000 items just to produce "and 9,990 more...". Since this only runs on the failure path the practical impact is low, but if Materialize is adopted above the count comes for free from List.Count and the problem disappears entirely.


What's working well

  • The GetExpectation() fix to call FormatValue(_expected) is the right change and gives symmetrical formatting for both sides of the message.
  • The IsEquivalentTo hint when references differ but contents match is excellent discoverability — this is exactly the kind of guidance that turns a confusing failure into a self-explaining one.
  • Test coverage is solid: different contents, same contents / different references, int[], List<int>, and truncation are all exercised independently.
  • The 10-item cap is a sensible default; the and N more... phrasing is clear.

The double-enumeration fix is the only blocking change needed. Everything else is in good shape.

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test post to verify auth

@codacy-production
Copy link
Copy Markdown

codacy-production Bot commented Apr 18, 2026

Not up to standards ⛔

🔴 Issues 1 high · 1 minor

Alerts:
⚠ 2 issues (≤ 0 issues of at least minor severity)

Results:
2 new issues

Category Results
ErrorProne 1 high
CodeStyle 1 minor

View in Codacy

🟢 Metrics 8 complexity

Metric Results
Complexity 8

View in Codacy

TIP This summary will be updated as you push new changes. Give us feedback

Addresses review feedback on #5619: FormatValue(value) exhausted the
enumerator, then SequenceEqualsNonGeneric tried to enumerate again.
For non-replayable sequences (yield generators, LINQ queries,
File.ReadLines, IAsyncEnumerable-backed sources) the second pass saw
an empty sequence and silently misreported "same contents, different
reference". Same hazard for _expected between SequenceEquals and
GetExpectation.

Enumerate both sides exactly once into List<object?>, format and
compare against the materialized lists, and cache the expected-side
format so GetExpectation reuses it. Adds a regression test using a
yield generator.
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up Review: fix(assertions): render collection contents in IsEqualTo failure messages

The previous critical concern (double enumeration of non-replayable sequences) has been fully addressed. The second commit materialises both sides exactly once into List<object?> before any formatting or comparison, which is exactly the right fix. The regression test with a yield return generator confirms the fix is exercised.

Previous Issues — Resolved

Double-enumeration bug ✅ Fixed. CheckAsync now calls Materialize on both value and _expected upfront, and all subsequent operations (format, compare, cache) work against the materialized lists. GetExpectation() reuses _cachedExpectedFormat rather than re-enumerating.


Remaining Minor Observations

String items inside collections are not quoted

FormatItems calls items[i]?.ToString() directly, so a collection of strings renders as [hello, world] rather than ["hello", "world"]. This is mildly ambiguous — null and the string "null" would look the same, and whitespace-only strings are invisible. Replacing the inner format call with a small helper that quotes strings would close this gap:

private static string FormatItem(object? item) => item switch
{
    null    => "null",
    string s => $"\"{s}\"",
    _       => item.ToString() ?? "null"
};

Then in FormatItems: parts[i] = FormatItem(items[i]);

Nested collections still show type names

Items that are themselves collections (e.g. List<int[]>) are formatted via ToString() inside FormatItems, giving back System.Int32[]. FormatValue handles this case correctly via recursion but FormatItems bypasses it. Not blocking, but it means List<int[]> doesn't benefit from this fix at the item level.

FormatValue is effectively only reachable from the non-collection fallback path

The static helper FormatValue is only called from GetExpectation() when _cachedExpectedFormat is null (i.e. the collection path was not taken — either a comparer was supplied, or one side isn't a collection). Its internal Materialize call on a non-replayable IEnumerable is technically safe in that path because the collection branch in CheckAsync also would not have run. However, the method now exists in the class but is functionally unused for the common case — worth a comment or the rename to FormatFallbackValue to make the intent clear.


What's working well

  • The fix is minimal and targeted — only the failure path allocates; the happy path is untouched.
  • _cachedExpectedFormat cleanly bridges CheckAsync and GetExpectation() without exposing mutable state.
  • SequenceEqualsLists uses object.Equals which correctly handles nulls on both sides.
  • The IsEquivalentTo hint for same-contents-different-reference is an excellent UX touch.
  • Test suite is thorough and includes the non-replayable-sequence regression.

The blocking issue from the prior review is resolved. The remaining items above are improvements worth considering but are not blocking merge.

…ns in IsEqualTo format

Addresses follow-up review on #5619:
- String items inside collections are now quoted so ["hello", "world"]
  is unambiguous — the literal "null" and a null element no longer
  look identical, and whitespace-only strings remain visible.
- Nested collections recurse, so List<int[]> renders as
  [[1, 2], [3, 4]] rather than leaking "System.Int32[]".
- Clarify via comment that FormatValue is only reached from the
  GetExpectation fallback path; the collection path in CheckAsync
  materializes once and calls FormatItems directly.

Adds StringArray_IsEqualTo_Failure_Quotes_Items and
NestedCollection_IsEqualTo_Failure_Recurses_Into_Items regressions.
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final Review: fix(assertions): render collection contents in IsEqualTo failure messages

All three rounds of feedback have been addressed in this PR. This is a solid fix that meaningfully improves test failure UX.

Previous Issues — All Resolved ✅

  • Double-enumeration of non-replayable sequences ✅ — Both sides materialised once into List<object?> upfront; _cachedExpectedFormat ensures GetExpectation() reuses the already-computed string.
  • String items unquoted inside collections ✅ — FormatItem now wraps strings in "" so null and the string "null" are unambiguous.
  • Nested collections leaking type names ✅ — FormatItem recurses into IEnumerable via FormatItems(Materialize(enumerable)).
  • FormatValue intent unclear ✅ — Comment added explaining it's the fallback path for GetExpectation() when the collection branch didn't run.

One Remaining Minor Inconsistency

SequenceEqualsLists uses object.Equals(actual[i], expected[i]) at the element level. For nested collections — e.g. int[][] where both sides have identical contents in different array instances — Equals falls back to reference equality, so SequenceEqualsLists returns false and the IsEquivalentTo hint is suppressed. The formatting is now correct (nested contents render recursively), but the discoverability hint doesn't surface for the nested case.

This is a pre-existing limitation of using EqualityComparer<T>.Default semantics and not a regression introduced here. Fixing it completely would require recursive sequence equality, which is a larger scope change. Worth a follow-up issue but not blocking.

Theoretical: FormatItem recursion on self-referential collections

FormatItem recurses into IEnumerable items via Materialize. A self-referential collection (e.g. list.Add(list)) would loop until a stack overflow. This is an edge case essentially impossible to encounter in an assertion argument, but worth being aware of if the formatter is ever exposed more broadly.

What's Working Well

  • The single-materialization design is correct and clean — failure path allocates once, happy path is untouched.
  • _cachedExpectedFormat bridges CheckAsyncGetExpectation() without leaking mutable state outward.
  • The IsEquivalentTo hint for same-contents-different-reference is excellent discoverability UX.
  • Test suite is thorough: 7 regression tests covering different types, references semantics, non-replayable sequences, truncation, string quoting, and nested collections.
  • String-before-IEnumerable ordering in FormatItem correctly prevents strings being formatted as char sequences.

The PR is in good shape. The minor nested-collection hint inconsistency is the only outstanding gap, and it is pre-existing rather than regressed here. Ready to merge.

intellitect-bot pushed a commit to IntelliTect/EssentialCSharp.Web that referenced this pull request Apr 21, 2026
Updated [TUnit](https://github.com/thomhurst/TUnit) from 1.35.2 to
1.37.10.

<details>
<summary>Release notes</summary>

_Sourced from [TUnit's
releases](https://github.com/thomhurst/TUnit/releases)._

## 1.37.10

<!-- Release notes generated using configuration in .github/release.yml
at v1.37.10 -->

## What's Changed
### Other Changes
* docs(test-filters): add migration callout for --filter →
--treenode-filter by @​johnkattenhorn in
thomhurst/TUnit#5628
* fix: re-enable RPC tests and modernize harness (#​5540) by @​thomhurst
in thomhurst/TUnit#5632
* fix(mocks): propagate [Obsolete] and null-forgiving raise dispatch
(#​5626) by @​JohnVerheij in
thomhurst/TUnit#5631
* ci: use setup-dotnet built-in NuGet cache by @​thomhurst in
thomhurst/TUnit#5635
* feat(playwright): propagate W3C trace context into browser contexts by
@​thomhurst in thomhurst/TUnit#5636
### Dependencies
* chore(deps): update tunit to 1.37.0 by @​thomhurst in
thomhurst/TUnit#5625

## New Contributors
* @​johnkattenhorn made their first contribution in
thomhurst/TUnit#5628
* @​JohnVerheij made their first contribution in
thomhurst/TUnit#5631

**Full Changelog**:
thomhurst/TUnit@v1.37.0...v1.37.10

## 1.37.0

<!-- Release notes generated using configuration in .github/release.yml
at v1.37.0 -->

## What's Changed
### Other Changes
* fix: stabilize flaky tests across analyzer, OTel, and engine suites by
@​thomhurst in thomhurst/TUnit#5609
* perf: engine hot-path allocation wins (#​5528 B) by @​thomhurst in
thomhurst/TUnit#5610
* feat(analyzers): detect collection IsEqualTo reference equality
(TUnitAssertions0016) by @​thomhurst in
thomhurst/TUnit#5615
* perf: consolidate test dedup + hook register guards (#​5528 A) by
@​thomhurst in thomhurst/TUnit#5612
* perf: engine discovery/init path cleanup (#​5528 C) by @​thomhurst in
thomhurst/TUnit#5611
* fix(assertions): render collection contents in IsEqualTo failure
messages (#​5613 B) by @​thomhurst in
thomhurst/TUnit#5619
* feat(analyzers): code-fix for TUnit0015 to insert CancellationToken
(#​5613 D) by @​thomhurst in
thomhurst/TUnit#5621
* fix(assertions): add Task reference forwarders on
AsyncDelegateAssertion by @​thomhurst in
thomhurst/TUnit#5618
* test(asp-net): fix race in FactoryMethodOrderTests by @​thomhurst in
thomhurst/TUnit#5623
* feat(analyzers): code-fix for TUnit0049 to insert [MatrixDataSource]
(#​5613 C) by @​thomhurst in
thomhurst/TUnit#5620
* fix(pipeline): isolate AOT publish outputs to stop clobbering pack
DLLs (#​5622) by @​thomhurst in
thomhurst/TUnit#5624
### Dependencies
* chore(deps): update tunit to 1.36.0 by @​thomhurst in
thomhurst/TUnit#5608
* chore(deps): update modularpipelines to 3.2.8 by @​thomhurst in
thomhurst/TUnit#5614


**Full Changelog**:
thomhurst/TUnit@v1.36.0...v1.37.0

## 1.36.0

<!-- Release notes generated using configuration in .github/release.yml
at v1.36.0 -->

## What's Changed
### Other Changes
* fix: don't render test's own trace as Linked Trace in HTML report by
@​thomhurst in thomhurst/TUnit#5580
* fix(docs): benchmark index links 404 by @​thomhurst in
thomhurst/TUnit#5587
* docs: replace repeated benchmark link suffix with per-test
descriptions by @​thomhurst in
thomhurst/TUnit#5588
* docs: clearer distributed tracing setup and troubleshooting by
@​thomhurst in thomhurst/TUnit#5597
* fix: auto-suppress ExecutionContext flow for hosted services (#​5589)
by @​thomhurst in thomhurst/TUnit#5598
* feat: auto-align DistributedContextPropagator to W3C by @​thomhurst in
thomhurst/TUnit#5599
* feat: TUnit0064 analyzer + code fix for direct WebApplicationFactory
inheritance by @​thomhurst in
thomhurst/TUnit#5601
* feat: auto-propagate test trace context through IHttpClientFactory by
@​thomhurst in thomhurst/TUnit#5603
* feat: TUnit.OpenTelemetry zero-config tracing package by @​thomhurst
in thomhurst/TUnit#5602
* fix: restore [Obsolete] members removed in v1.27 (#​5539) by
@​thomhurst in thomhurst/TUnit#5605
* feat: generalize OTLP receiver for use outside TUnit.Aspire by
@​thomhurst in thomhurst/TUnit#5606
* feat: auto-configure OpenTelemetry in TestWebApplicationFactory SUT by
@​thomhurst in thomhurst/TUnit#5607
### Dependencies
* chore(deps): update tunit to 1.35.2 by @​thomhurst in
thomhurst/TUnit#5581
* chore(deps): update dependency typescript to ~6.0.3 by @​thomhurst in
thomhurst/TUnit#5582
* chore(deps): update dependency coverlet.collector to v10 by
@​thomhurst in thomhurst/TUnit#5600


**Full Changelog**:
thomhurst/TUnit@v1.35.2...v1.36.0

Commits viewable in [compare
view](thomhurst/TUnit@v1.35.2...v1.37.10).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=TUnit&package-manager=nuget&previous-version=1.35.2&new-version=1.37.10)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This was referenced Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant